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PREFACE 


« This study was conducted under Project 1123, USAF Elying Training Development; 
Task 112302, Instructional Innovations in USAF Flying Training. 
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; . I. INTRODUCTION vs 


Any:time two or more aircraft move from one place to another, the most expeditious and efficient 

way to do so is in groups, provided the group is ordered, disciplined and performs predictably. A 

competently led formation is the best method because it allows application of the*best leadership and 

_ provides a way to move a number of aircraft through limited airspace without overcrowding that airspace or 
“gBrurol facilities. Additionally, it provides a redundant navigation and communications capability for all 
Porat in the flight. Flying in close proximity to another aircraft during maneuvering flight is One of the 
most\ demanding requirements of military aviation. As a formation member, the pilot must be disciplined 
and aggressive with quick and precise reactions. Formation training is unique in that it has always been 
conducted exclusively in the aircraft. This is a stressful situation which most assuredly does not provide the 
conditions most conducive to learning. Flight simulators offer an alternative training environment which 
this study was designed to investigate ‘ 


' 

‘ The, effectiveness of flight simulatdrs has repeatedly been demonstrated in instrument training 
(Valverde, 1968; Kelly, 1970). The advent of practical visual flight simulators, and their demonstrated 
effectiveness by the commercial airlines (Gjbson, 1969) has greatly expanded the role of simulation in 
flving tfaining. Prompted by this expansion, the Air Force developed a simple’ visual flight simulator that 
allows a$tudent to practice formation flying prior to instruction in an aircraft (Wood, Hagin, O'Connor, & 
Myers, 1972). A previous study, in which an attempt was made to provide limited jnformation practice in 
thy! F100/151 Fixed Gunnery Trainers, suggested that such a trainer would be effective (Pfeiffer, 1963). 
Pteiffer's study defonstrated improvement in ability to maintain formatfon position in the simulator in 
spite of equipment limitations that prevented the student from flying closer to the lead aircraft than 2,000 
feet. However, the effect of this training on subsequent formation flying in an aircraft was not studied. 


In the present research, the investigator sought to demonstrate the transfer of simulator practice to 
actual aircraft flight for the wing aircraft component of the formation flying task. Two studies were 
conducted in the T-38 phase of United States Air Force Undergraduate Pilot Training (UPT) using a 
prototype Formation Flight Trainer (FFT). These studies were designed to provide a direct answer to an 
applied question: “Can the FFT provide enough training to justify purchasing them in“ quantity for 
inclusion in UPT?” Based on projected unit cost of producing FFTs, pilot production rates and estimated 
training cost, it was determined that cost-effectiveness would be achieved if the FFT could reduce 
formation practice in an aircraft by at least two hours. * 


instructional potty of the USAF Air Training Fommand (ATC). The materials, procedures, etc., were the 
sane for both studies. The principal differefce TA the two studies was the prior experience in formation 
flying of the students. 


The second study is essentially a ng of the first study, brought about by a change in 


In study 1, the students were essentialf-ngtve to formation flying. They had flown approximately 
three demonstration sorties in the T-37 a aining airplane), These sorties were composed primarily 
of demonstration by the instructor pilot. In fhe Second study, the students had flown approximately seven 
sorties in T-37 formation. At the end of thes@ seven sorties, the student was required by the training 
syllabus fo be able to fly formation without the i&structor pilot accompanying him in the airplane 


il. METHOD ~ STUDY I 


Subjects 


. Seventy students were selected for this: study from UPT classes 73-08 and 73-09 at Williams AFB, 
Arizona, The cTass¢s wré divided into three Ag | u hike for the T-38 formation phasgapf UPT. At this 
Stage of training, the students had completed a yxgmately 82.5 hours of flying training in the T-37 
ai.craft and 30 hours in the T-38. There was no reason to-believd that these classes were different from any 
other class of UPT students and, therefore, they wefe considered a sample from the population of UPT 
trainees. The sample was restricted to United Rare eiieane without flying experience prior to enteriig 


UPT 
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ATC planning ts to develop uP for the middle 68% of the trainee distribution and to individually 
manage the 16% low ability and the 16% high ability students (Mission Analysis Study Group, 1972). To 
assure that the samples for this study would provide data compatible with this philosophy, the students in 

“each flight of the classes involved were ranked on their. flying ability, as defined by the average of all 
checkride scores, at the end of the T-37 phase of training. Two students from the low 16% and two 
students from the high 16% were randomly assigned to each of three groups for each class. An additional 
eight students randomly selected from the middle 68% of each class completed the groups. Comparison of 
the nean scores of the groups’ final T-37 scores indicated that they were not significantly different (p > 
O5) in flying ability. In class 73-09, two students were eliminated from training on their contact checkgide 
subsequent to being selected and prior to entering the formation phase. Therefore, a total of 70 subjects 
participated in the study ; 


Equipment ,? 


a 

The FFT & a stimulation system which proyides a realistid two-sireraft formation flight ‘situation 
(Figure 1). The trainer provides the student with a wide angle projected television picture of a T-38 lead 
areraft that is continuously variable in range relative bearing and relative altitude, The picture of the lead 
aireraft_ can be commanded by an instructor pilot (IP) to perform standard maneuvers while the student 
ulttempts to maintain: posiuion in tormation by control actions from his own simulated T-38 aircraft 
cockpit, A detailed description of the FFT can be found elsewhere (Wood et al., 1972). 
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Figure 1. Formation flight trainer. 


Procedures 


The three comparison groups of the complete random design are depicted in Table |. Treatments 
were ratdomly assigned to the groups as follows: Group A was the FFT-trained or experimental group: 
Group B was a limited training group: and Group C was an aircraft or UPT-syllabus trained group. Previous 
expenence with the FFT indicated that five sorties was a reasgnable and useful number for an average UPT 
student dnd, therefore. it was decided for this study to use fiv®50-minute FFT sorties, in a block, prior to 
aircraft practice. Sorties length of 50 minutes was selected to conform to current UPT practice and block 
presentation ‘was selected because of its demonstrated effectiveness in instrument training in flight 
simulators (Wilcoxson, Davy, & Webster, 1954; Reid, Hagin, & Coats, 1970; Woodruff, Smith & Morris, 
1974). Each of the five FFT sorties was instructed by a different [P in an effort to control for IP 
ditferences in-experience or ability. The IPs were allowed to use any sequence of maneuvers and 
instructional technique they desired ( 


Table |. Number and Type of Flying Sorties by Treatment Group 


Sorties 
é 5 Aircraft . Aircraft 
: Orientation Aircraft Study+Data 
; Groups FFT F51-01 Training Ride . 
\. FFT 5 | 0 I 
B. Limited Training aa 0 | 0 ¢. 4 
C) UPT Syllabus 0 Mil} re 1 


Subsequent to flying the FFT sorties. the A group flew a sortie in the T-38. The purpose of this sortie 
was to allow the student to put what he had learned in the simulated fight environment into the airplane 
context. Either a sixth trainer IP, or the trainer IP who had instructed the first FFT sortie, flew this aircraft 
sortieewith the student. All maneuvers were demonstrated and differences between the T-3% and FFT were 
emphasized. The day following the aircraft orientation mde, the student flew a T-38 checkride and his 
abslity to fly the airplane through nine basic maneuvers was evaluated 


Group B regeived only an atreraft orientation ride. All maneuvers were demonstrated and explained 
during the onentation ride. Thus, on the following day when a student was asked to alsempt a maneuver on 
a data checkride, he at least had been shown the desired performance 


Fach group C student fléw the first three aircraft sorties as if they were not part of a study. The first 
Sortie was essentially the same as the one flown by groups A and B, except it was flown with the student's 
normally assigned IPs. The day following the third atreraft sortie, the student flew the data checkride 


Instructor Pilots 


Fight TPs from the 7th Blying Training Squadron were instructors in FFT and the aircraft 
Onentation nides for groups AMaurd, B. The IPs had approximately a week to fly the FFT and discuss its 
capabilities wth other IPs angestuds personnel prior to the start of the study They were selected trom 
dilferent Mghts according: to thet willingness to tuke on an additional duty and their supervisors’ approval 
of -heir reduced availability for regular fying duty There is no evidence to suggest that these IPs yere not 
representative of all [Ps at Williams AFB, Arizona 


Performance Assessment 


An assumpnion was made that the most reliable assessment of student ability would fe provided by 
the expertenced and highly standardized squadron “check section” pilots using a gradingssystent simular to 
and*compatible with the existing grading system, Check section pilots are trained observers 4vho regularly 
fle approximately five checkrides a week Ten check pilots were used with all but one IP flying at least five 
data checkrides. One check pilot Mew as few as tour checknides and one flew thirteen checkrides. Attempts 
to control the number of students from each group that each check pilot flew with were unsuccessful 

‘ \ 


The grading seal weed ‘was an expansion of the existing UPT grade scale. The existing scale measures a 
student’s ability to fly a maneuver as follows: i* 


] 


Unable to Accomplish (U): The student lacked sufficient knowledge, skill, or/ability to perform the 
element, operation, maneuver, or task without ashistance. 


Fair (F) The student performed the element, operation{ maneuver, or task but 
made some false starts, repetitions, or minoy errors of omigsion or 
commission. | 

Good (G): The student performed the element, operdtion, maneuver, or task 
correctly, with little hesitation. 


‘ 


Excellent (E): . The student performed the element, ope§tion, maneuver, or task 
correctly, quickly, and efficiently. 


Since thé students were being evaluated at a very early stage of training/it was anticipated that they 
woyld all score in the U or F categories. In an effort to increase sensitivity, an expansion to a 12-point scale 
was devised which allowed an IP to score performance in the upper half, lower half, or middle of each of 
the existing scale ufits. 


& 


Instructor had to assume control almost immediately to avoid collision. 


The 12-point scale was defined as follows: 


Instructor eventually had to assume control as performance deteriorated. 

Instructor never assumed control; however, performance was still unsatisfactory. 
Performance very rough; however, instructor found that verbal assistance corrected problem. 
Performance rough — minimal verbal assistance would correct problem. 


Performance rough; however, no verbal assistance necessary - practice should improve 
performance. . 


Performance somewhat smoother than an F student; however, becomes rough after short tirke. 
Performance somewhat smooth but continuously passes through desired position. 
Performance smooth qeviations from desired position last several seconds. 


Performance very smooth — after deviations aircraft returned to position quickly. 


‘ 


Performance very smooth — deviations are small and aggressively corrected. 


No deviations noted. Perfect position muintained. 


A precisely defined data checkride profile was developed to insure that all students were evaluated in 
the same way. The profile consisted of the following nine maneuvers straight and level, shallow bank turns 
(15° — 20°),-medium bank turns (30° — 40°), steep bank turns (60° 90°), route, crossunder, echelon 
turns (45° bank), turning rejoin and straight ahead rejoin. The order of the nine basic formation maneuvers 
was precisely defined to prevent students from having diferent amounts of practice prior to evaluation. 
When the profile called for a turning maneuver, one was flown in each direction prior to check IP assigning 
a grade. The initial turn could be in either direction. In this way, the check IP was able to keep the aircraft 
in the assigned flying area with. minimum interference ta the profilé. When operational restrictions 
prohibited the flying of the next maneuver as defined bv thé profile, the IP took control of the aircraft 
until the profile could be exectited. Check, IPs were asked not to instruct on any maneuver until after 
checkride completion. The lead aircraft wfs always flown by an IP rather than the student to insure that 
lead was as stable as possible for afl students flying wing. The data checkride was not part of the normal 
training program and was inserted at the appropriate time according to which group the student was 
assigned. Upon completion of the data checkride, the remainder of the student formation training, was 
performed as Pantie by the training aytlabus, 


The grade for each student’s data checkride was derived by multiplying cach maneuver grade by a 
weight extracted from the oferational ATC checkride. These weights account forthe varying difficulty and 
importance of the maneuvefs to the entire operational task. The nine maneuver grades were then summed 
and transformed to standard scores with a 0 = 10 and X = 50 (Guilford, 1965). The equation used was: 


10 X, 10\ M, 50 | = T where: 
0 4, 


0 
Mean grade assigned by a particular cleck pilot. 2 
Standard deviation of grades assigned by a particular check pilot. 


Observed grade for one student by a particular check pilot. 


"This standard score transformation was performed to partially compensate for the subjective nature of the 
observations combined with the effect of ten check pilots making an unequal number of observations» 


y Il, RESULTS — STUDY 1 
Erg 
rs i , 
The data checkride grades for th $hrget groups were analyzed by a one-way analysis of variance 
(Table 2). The differences between th® three froups were statistically significant F (2,66) =742, p< 0S. 
Calculation of ah Omega Square indicates that the treatment effect accounted for 15.7% of the variance 


Table 2. Analysis of Variance: Aircraft 
Performance Scores — Stady | 


Source ; Ms 


Between Groups 598.26 
Within Groups 80.66 


Total 


*p <.OS. 


Subsequent to the analysis of variance, an “a posterion @st" {Tukey's HSD), was performed to 
ascertain significance for between group comparisqns (Kirk, 1968). As indicated in Table 3, both the 
FFT-trained (Group A) and the UPT syllabus-trainéd (Group C) groups scored higher than the limited 
training gfoup (Group B). The difference in both comparisons was significant at the 05 level. The mean for 
the UPT syllabus-trained group was higher than the mean for the FFT-trained group but the difference was 
not found to be significant at the OS level. \ 


Tablc 3. Differences Among Means of Performance 
Scores — Study I 


Groups ‘ Xe Xa Xe 
' 


Xp, (Limited Training) : 6.66* » 10.04* 
X, (FFT) 2 “ tt 3.38 
X,. (UPT Syllabus) 


fl 


= 69 


“ 


a 


pince the“nine maneuvers have different degrees of importance to the flying task, weights were 
assigned to the maneuvers andthe one-way analyses of variance was repeated using the weighted data. The 


weights were extracted from the current UPT two-ship checkride,as-défined by ATC. This analysis revealed” 


results comparable to the previous analysis F (2,66)-= 74T. p < .05 (Table 4). The between group 


comparisons are presented in Table 5. . 
f 
er ge Table 4. Analysis of Variance, Weighted 
Scores — Study I 
rs Source DF MS F 
Between Groups a - $99.26 7.42* 
. Within Groups =, 6 ~~ 80.66 
, Ot . 
Total da ara 
“ P *4 
i (pe 
4 ay . 
Table 5. Differences Between Group Means vf Vom 
Weighted Scores — Study I : A A 
(Formation Flight Trainer Application to UPT) : 
Differences Between Group Means 
Groups . A 8 c 
A. FFT £3) pe Sumas 3 . 
B. Limited Training al -- 
C, UPT Syllabus (3 Sorties} ~., 3.38 10.04* -- 
‘ 
N= 69. 
*p <.05. 
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IV. METHOD - STUDY I 


The deseription of method for study II ist the same as Study | with the execptions described 
subsequently 


Subjects A \ 


Forty-eight students were selected fpom UPT class 74-06 at Williams AEB, Arizona. The class was 
divided into three study groups for the T-38 formation phase of UPT. At this stage of training, the students 
had completed approximately 90 hours of fying training, eight of Which were formation in the T-37 
aircraft and 30 hours in the T-38. The sample was selected and assigned to treatments according to the 
procedures used in study 1. 5 
: \ 

Instructor Pilots \ 

Eight [Ps ftom the 97th Flying Training Squadron were instructors \in the FFT and in the aircraft 
orentation rides tor the A group. None of the IPs from study 1 pafticipated in Study I. They were selected 
from different flights according to the procedures established in study [. 


Performance Assessment 


In an effort to gain better control of the evaluation procedure, the use df\pilots from the squadron 
cheek section was abandoned. The work load of the check section necessitates ihe use of a relatively large 
nu nbet of observers. Even though check section is highly standardized, it was devided that better control 
could be obtained by having a smaller number of observers trained to observe facts related to this study, 


: ia Ln: By 
-Y 
ee \ , 


® : : é _ ; e 
i Equally, as-important, this procedure had less impact hi the squadr6n’s training mission. The number o 
check pilots was reduced to four and line/ins{ructors were used. Aftér the start of the study two of thes 
pi'ots had to be replaced. Therefore, thesstudénts re evaluated by six IPs. The students’ data checkrid 
scores were derived according to the procedures developed in study I. . 
: " 838 a, 
< 7 ' . \ 
; V. RESULTS - STUDY II 
The data checkride scores for the three groups were analyzed with a one-way analysis of variance. Ph 
* differences between the three groups were statistically significant F (2,39) = 5.33, p ¢ .05 (Table 6) ‘ 
Calculation of an Omega Square indicates that the treatment effect accounted for 17.5% oe variance. Z 
, $ \- 
: Rs ‘Table 6. Analysis of Variance, Performance 
: a! he Scores — Study II 
‘ \ Source OF MS : F 
Between Groups 2 449.5] 5,33* 
Within Groups 39 ~ 8423 
. M a 
¢ Total 41 1! p ; K+ 
F eS a ee ee ee bs og 
ey *p< 05. f Ea 
< . a es 
: Subsequent’ to the analysis of variance, and “a posteriori test” (Pukey’s HSD) was performed t« 
ascertain significance for the between group comparisons (Kitk, 1968). As indicated by Table 7, both th 
FFT-trained (Group A) and the UPT syllabus-trained (Group C)\'groupg scored higher than the limite: 
training group (Group B). The difference in both comparisons Was signifigdnt at the .05 level. The mean fo 
the FFT-trdined group was higher than the mean for the UPT syllabus-trained group but the difference wa » 
~ not found to be significant at the OS level. : : . 
#3 i Table 7. Differences Among Means of Perforgance 
= : Stores — Study I al : x 
2 F Foe ae Sa, - 
Groups. 8 c A 
B. (Limited Training) = 439 992% JLB 
' C.-(UPT Syllabus Trained) = 53.41 +: -: 1.90 
A. (FFT) 795.31 -- . 33 : 
: Se be 
\ : N=42, - : “ * ‘ 
\ - . *p<.05. : . ; / 
- > \ 
4 The analysis of the weighted scores, as in study I, revealed results comparable to the unweighted f 
(2,21) =5.33,p | : ) 
, ‘ a | . 
Table 8. Analysts of Variance, Weighted Scores — \ 2 
Study Il . ‘ ( . 
Source DF Ms Fe . 
Between Groups ERS 449'52 - *5.33* 


Within Groups 39 


Table 9. Differences Between Group Means Weighted Scores — 
Study Il 


; Differences Between Group Means 


; § Groups A B- Cc 


A. (FFT) = 55.31 - 
B. (Limited Training) = 43.49 1 {go -* 
C. (@PT Syllabus Trdined) = 53.41 1.90 9.92% 


N= 43. 
*p<.05, 


Vi. DISCUSSION — STUDY | AND STUDY II 

The present research was conducted to determine transfer of station ina formation flight simulator 
to formation flying in the, aircraft. Evidence in support of positive transfer was obtained in these two 
stuaies by comparing the mean pétformance of students in the FFT-trained groups with the mean 
performance of students in the limited training groups. Students trained in the FFT scored significantly 
better than the essentially untrained students, indicating that simulator training resulted in improvéd 
performance in the T-38 aircraft. Likewise, comparing the means of the UPT syllabus-traingd groups with 
the limited training groups means confirms the obvious, that practice on a task improves performance on 
that task. However, differences between FFT-trained groups and the UPT syllabus-trained groups fail to 
attain statistical signifigance, These comparisons lead to the conclusion that both training methods are 
effective at this early stage of the student’s skill uple Since student performance does not indicate 
that one training method is obviously superior, | en other factors, such as cost of training, should be 
included in selection of a training program. ' e : 


A post hoc analysis of group equivalence was. performed using scores from the contact phase of 
training. The contact checkride was the last activity participated in prior to starting the formation training 
" phase and shoutd provide information about differences jn flying ability at the start of the study, An 

analysis of variance failed to show any statistically significant differences in these contact scores. Although 
these data are indicators of flying skill, they could not be used in the analysis as a pretest because the scores 

come from different flying tasks. These data, however, do lend supporLto the conclusion that the observed 
silterenices in the studies were treatment effects. 


Study” rT was conducted for two reasons. The first was to replicate study I in alee to increase, 
ssaltlbenca in the results. The second was to determine the effect of a change in the UPT syllabus. Results 
' of the*secondsstudy did not reveal any change from study I, which permits some interesting speculation. 
Th: limited training groyp for study If was expected’ to score higher on the data checkride than the limited 
training group for study I’, since they had received twice as many formation trianing flights in the T-37. But 
scores obtMined in study II do not indicate a higher’skill entry level for T-37 formation-trained students. 
Hoaever, it is possible that the overall level of proficiency at the conclusion of T-38 formation is higher 
with the additional training, but these investigations did not attempt to answer this question. 


A primary consideration in understanding the contribution to training of any simulator is the way. in 
which the device ig used. The purpose of these studies was to examine the contribution of: the simulator per 
se. In order to avoid biasing the outcome of these studies with transfer not directly attributable to the 
simulator, a conservative approach was taken in the development of instructional strategies. Such things as 
assuring that each student was instructed by a different IP each day and allowing the IP to teach according 
to his own plan areyexamples of this approach. It: is suggested that the development of effective 
instructional strategies, the definition of the amount of useful simulator practice and determination of how 
formation simulator jcan best complement aircraft instruction will result in a substantial increase in the 
FFT’s'training effectiveness. 


Results from these studies, especially in light of their repeatability, provide conclusive Daisies that 
the formation simulator is an effective training device. 


Future research with this prototype simulator should define more effective instructional uses and 
es sequences, The resultant training program will undoubtedly provide more efficiently trained 
ots. * 
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